With evolution of an image capturing apparatus and development of an image processing technique in recent years, it is desirable to automatically analyze and utilize moving images. There is a possibility that in techniques analyzing moving images, estimation of motion of an image-captured person is useful in many application fields such as a monitoring camera, a robot vision, or the like where human beings are involved. When estimating motion of a person, the movement of a head of a person including a face is extremely important information. Techniques for estimating motion of the head of a person by recognizing the head through analysis of a moving image are disclosed in some prior work as further detailed below.
A method for recognizing the head of moving object considered to be a person in a moving image by detecting a waist portion (i.e., concave portion) of the outline of the head has been proposed (for example, Y. Tian, et al., “Absolute Head Pose Estimation from Overhead Wide-Angle Cameras,” Proceedings of the IEEE International Workshop on Analysis and Modeling of Faces and Gestures (AMFG '03), pp. 92-99, 2003). Also, methods for recognizing the head by matching a shape, and the like, have been proposed (for example, refer to Patent Document Japanese Unexamined Patent Publication (Kokai) No. 2002-157596 which discloses a method for performing real-time tracking of a face based on color distributions included in detected face patterns, and applying moving changes for face detection. Patent Document 2, Japanese Unexamined Patent Publication (Kokai) No. 2002-203239, discloses an algorithm for identifying a potential face based on detected face characteristics (eye, eyebrow, nose, and mouth). Patent Document 3, Japanese Unexamined Patent Publication No. 2002-203239 discloses face detection equipment including face similarity calculation means for computing similarity of face-likeness by comparison with face information stored in a storage section.
Also, Patent Document 4, Japanese Unexamined Patent Publication (Kokai) No. 2004-178229, discloses a man existence location detector which includes a segment-like vote section, which is a vote template modeling the shape of person's head and used to widen the detection range. Patent Document 5, Japanese Unexamined Patent Publication (Kokai) No. 2004-206656, discloses a detection device for stochastically calculating human body likeness by using the probability of a body part candidate's part-likeness, and the interrelation between parts. Patent Document 6, Japanese Unexamined Patent Publication (Kokai) No. 2004-53324, discloses a collision safety controller for automobiles which extracts a passenger's head based on a profile configuration of the passenger's head.
Patent Document 7, 8, 9, 10, 11, 12, 13, and 14, Japanese Unexamined Patent Publication (Kokai) No. H10-274516, Japanese Unexamined Patent Publication (Kokai) No. 2001-216518, Japanese Unexamined Patent Publication (Kokai) No. 2001-243466, Japanese Unexamined Patent Publication (Kokai) No. 2003-141541, Japanese Unexamined Patent Publication (Kokai) No. 2004-78778, Japanese Unexamined Patent Publication (Kokai) No. 2005-18654, Japanese Unexamined Patent Publication (Kokai) No. 2004-272933, Japanese Unexamined Patent Publication (Kokai) No. 2001-291108, respectively, disclose techniques for estimating motion of a head under the condition that a head or a face can be extracted. Patent Document 7 discloses a device for detecting the sense of face by detecting the center position of a face region akin to an ellipse, created by extracting description regions such as an eye, a nose, a mouth and the like.
Patent Documents 8 and 9 disclose devices which perform matching to templates of face images with different directions. Patent Document 10 discloses a device simultaneously displaying information regarding a face image at the time of registration and information regarding a face image being input at the time of recognition. Patent Document 11 discloses a driver monitoring device including determination means for determining the sense of the face based on the differences between images collected serially.
Patent Document 12 discloses a device for detecting a looking-aside condition by detecting optical flow. Patent Document 13 discloses a device for determining which sense of face matches most closely to the face in a monitor image, by preparing face images, such as a transverse-plane face, a leftward face, and a rightward face and by using canonical discriminant analysis. Patent Document 14 discloses an image processing device having database creation means storing average face images which are made from three-dimensional average face models and which correspond to various angles.
The method described in Non-patent Document 1 has a problem such that recognition of the head is difficult when accuracy of the extracted moving object is reduced because of the characteristics of the image or the background, since a waisted portion of the outline was not detected. For example, if the background includes colorful and complicated texture, since the outline of the moving object is not smooth but has lots of dents, it is difficult to determine the correct waisted portion of the outline by using this method. Also, for example, in the case in which both hands are put on the head, or in the case in which a person is wearing high collared clothes, dents appear in places other than the intended place, it is difficult to determine the correct waisted portion of the outline by using this method.
In the methods disclosed in Patent Document 1, 2, and 3, it takes a long time to match since there are many parameters. Also, in these methods, it is not possible to obtain stable matching results when the outline is not smooth. Also, in the methods disclosed in Patent Document 4, 5, and 6, it is difficult to recognize the head of a person when the outline is not smooth.
The devices disclosed in Patent Document 7, 8, 9, 10, and 11 are not capable of detecting motion of a head, in the case in which it is difficult to recognize each part of a face due to low resolution of the image, in the case in which a person's face is turned away from the image pick-up side of a camera by more than 90 degree, and the like.
With regard to the device for detecting optical flow described in Patent Document 12, it is possible to compute this when the moving image is obtained under good conditions, such as using an in-vehicle camera, however, it is not possible to compute this when the face is small, when characteristic parts such as eyes, a nose, and the like, cannot be extracted as the person is facing obliquely away from the image pick-up side, or when the background is colorful and its texture is complex, and the like.
In the devices disclosed in Patent Document 13 and 14, a problem about what angle of the interval the data should have arises when the data is prepared. When the interval of the angle is small, matching takes a long time, and when the angle of the interval is large. the accuracy of detection deteriorates. Also, incorrect detection occurs due to the difference between a prepared face image and an input face image. Furthermore, the matching accuracy also deteriorates when a person is wearing glasses, a hat, and the like. Furthermore, the matching accuracy with the prepared date deteriorates when the extracted moving object is rough due to the characteristics of the image and the background.
As described above, in the conventional techniques, accuracy of recognition of a head and estimation of motion of a head are not good if the contour is not smooth due to the deteriorated accuracy of extracting a person's image. For example, in the conventional techniques, under the condition in which a monitoring camera is installed in the store of retailer, the outline of a person can not be extracted since the background (shelves for goods) is colorful and its texture is complex.
It is therefore an object of the present invention to provide a head detecting device, a head detecting method, and a head detecting program which can solve the above-mentioned problems. The object is achieved by the combinations of characteristics of independent claims in the scope of claims. Dependent claims also regulate advantageous specific examples of the present invention.
The present invention provides a head detecting apparatus including a foreground extraction section for extracting a foreground region in which a person is captured, from an input image, a first main axis computing section for computing a main axis of the foreground region, a head computing section for computing a head region included in the foreground region as a part thereof based on the main axis of the foreground region and a shape of the foreground region, and an ellipse determining section for determining an ellipse to be applied to a person's head in the input image based on a shape of the head region; a head detecting method using the head detecting apparatus, and a head detecting program to cause a computer to function as the head detecting apparatus.
The concept of the present invention is not the enumeration of all the necessary characteristics of the present invention, but subcombinations of the characteristics thereof may be also accepted as the invention.
According to the present invention, an ellipse can be appropriately applied to a person's head regardless of the accuracy of an extracted foreground region.